Method, device and system for intracavity probe procedure planning

ABSTRACT

A method (and imaging system) for planning a medical intervention on patient is provided that involves an intracavity probe and an imaging dataset of the patient. A view-type is selected from a defined set of view-types. A virtual field of view of the intracavity probe corresponding to the selected view-type is determined. A virtual intracavity image is rendered for display, wherein the virtual intracavity image is based upon the imaging dataset of the patient and the virtual field of view of the intracavity probe. The virtual field of view can be based upon segmentation of an intracavity path of the probe and at least one anatomical structure or possibly based on user input. In embodiments, the virtual field of view can be based upon probe parameters computed in accordance with a pre-defined set of rules for the selected view-type. The probe parameters can be computed by evaluation of a cost function expressed by the pre-defined set of rules for the selected view-type. Other aspects are described and claimed.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application claims priority from U.S. Provisional App. No.62/774,800, filed on Dec. 3, 2018, (Attorney Docket No. MEN-004PROV),herein incorporated by reference in its entirety.

BACKGROUND 1. Field

The present disclosure relates to the field of medical interventions.More particularly, the present disclosure relates to a method forpreparing or planning a medical intervention, such as a structural heartprocedure.

2. State of the Art

Structural heart disease covers a wide range of cardiac conditions,including valvular heart disease, arrhythmia, and defects in themuscular structure of the heart. The disease may be congenital, as wellas acquired. As the western population ages, acquired disease, such ascalcific (senile) aortic stenosis and mitral regurgitation has increasedin importance. The past two decades have seen a revolution in thetreatment of structural heart disease with transcatheter therapies beingdeveloped, for instance, for valve repair and replacement, closure ofdefects such as ASD (atrial septal defects), and isolation of the leftatrial appendage to reduce embolic risk in atrial fibrillation. Patientswho previously could only undergo high risk surgical procedures or werecompletely inoperable can now be treated with a transcatheter approachperformed in a catheterization laboratory, often with only a one-nightstay in the hospital.

When it comes to structural heart disease treatment, computed tomography(CT) plays an important role in pre-operative transcatheter procedureplanning. CT provides the physician with accurate three-dimensionalinformation of the heart structure and possible surrounding structures.For instance, in transcatheter valve replacement or repair, CT has animportant role in device selection by determining the anatomy andgeometric measurements of the for instance the valve annulus asdescribed by Thériault-Lauzier et al, “Computed Tomography forStructural Heart Disease and Interventions”, Interventional Cardiology(2015) September; 10(3): 149-154, where it is concluded that sizing ofthe patient anatomy and visual anatomical assessments are important forusing the appropriate device and making the correct treatment decisions.

Transcatheter procedures, for instance transcatheter aortic valvereplacement, are performed in a catheterization laboratory in whichX-ray is the fundamental imaging modality. A majority of thesetranscatheter procedure are performed under the guidance oftransesophageal echocardiography (TEE). Due to the TEE's high temporalresolution, possibility to assess blood flow and different tissueresponse as compared to X-ray as used during a transcatheter procedure,TEE is a complementary imaging modality. For instance, TEE is able toassess the valve leaflets as well as the valve leaflet motion during thecardiac cycle.

TEE is a semi-invasive technique which requires the insertion of TEEprobe, being a tube of approximately 10 mm in diameter, in theesophagus. TEE is vital for guiding and monitoring the entire process oftranscatheter heart valve procedures. For instance for proper placementof a mitral clip, coaxial alignment of the catheter towards the annulusis crucial for valve deployment, alignment of the catheter with anchorsto be positioned in the tissue is crucial for devices needing anchors ordevices that need to puncture tissue at pre-set location to guaranteetreatment efficacy. All these locations are monitored and guided bymeans of TEE.

To perform TEE accurately, knowledge of the procedure and anatomy isneeded for the echocardiographer and (interventional)cardiologist/surgeon pre-procedurally. The recommendation stated byCahalan et al. in “American Society of Echocardiography and the Societyof Cardiovascular Anesthesiologists” (Anesth Analg. 2002 June;94(6):1384-8), describe basic recommendations on the appropriate use ofperi-operative (during the procedure) TEE, with the intent of improvingoutcomes with evidence-based use of TEE. In addition, within theserecommendations the description of 8 additional views on top of the 20currently used views are described as a response to the newer upcomingtranscatheter heart procedures. As described in these recommendationsthe clinical indication for TEE should be the primary determinant ofwhich views are obtained first as well as the level of detail that isobtained from each view. It furthermore describes that the positioningof the TEE imaging device to obtain certain views is different perpatient because of individual variation in the anatomic relationship ofthe esophagus to the heart. For example, in some patients the esophagusis adjacent to the lateral portion of the atrioventricular groove,whereas in others it is directly posterior to the left atrium. Shiota etal, “Role of echocardiography for catheter-based management of valvularheart disease”, Journal of Cardiology 69 (2017) 66-73, also states thatit is important to realize that additional images, beyond the described28 views, may be necessary to comprehensively image specific structurese.g. for transcatheter heart procedures alignment of the catheter withcardiac structure to implant cardiac device. In addition, the degree ofrotation of the transducer and additional manipulation such as right orleft flexion, anteflexion or retroflexion, and turning of the probe maybe required in individual patients to achieve optimal TEE images.

It is well known that the quality of images obtained by TEE, and howthey are aligned with the device that is deployed, strongly relies onthe experience of the echocardiographer. It is therefore crucial thatthe echocardiographer holds an accurate knowledge of the anatomy andfunction of the heart structure and implantable device. Since TEE is ahighly user-dependent imaging modality, proper steering of the probe,spatial location, accurate knowledge of the anatomy and function theheart structure and implantable device play a vital role. Furthermore,complications such as small bleedings, chocking, cardiac arrhythmias andesophagus bleeding can occur during TEE imaging.

TEE simulation techniques are available with the aim to improve theskills of the echocardiographers. US patent application 2009/0162820discloses an education simulator for TEE, which includes a phantom whichmimics the human upper body and a predefined heart model. EP 2538398discloses a method that uses a three-dimensional model based on multipleCT images to simulate a TEE procedure for training TEE operators. BothUS2009/0162820 and EP 2538398 are aimed to provide TEE simulation foreducation purposes.

The vascular anatomy of patients who undergo a transcatheter heartprocedure is deviating from normal population and huge deviationsbetween patients are presents. To allow pre-operative planning withinthe setting of structural heart procedure for TEE imaging, patientspecific volumetric image data is required. Further to support in thedesired TEE image view, patient specific anatomical landmarks identifiedwithin the volumetric image dataset is required.

There is thus a need for a system that enables physicians (e.g. theechocardiographers) to plan TEE imaging for a specific patient. Thedisclosed method will also predict optimal parameters for obtainingparticular standard views (for instance “4 chamber”). Without thesepredictions, physicians need to search for the optimal orientationduring the procedure. The disclose method will thus lead to shorterprocedures and reduced patient risk.

SUMMARY

It is thus an object of embodiments herein to provide a method ofplanning a medical intervention on patient that involves an intracavityprobe, such as a TEE or intravascular probe. The method uses an imagingdataset of the patient. The method includes selecting a view-type from adefined set of view-types, determining a virtual field of view of theprobe corresponding to the selected view-type, rendering for display avirtual intracavity image based upon the imaging dataset of the patientand the virtual field of view of the probe.

In embodiments, the virtual field of view can be based upon segmentationof an intracavity path of the probe and at least one anatomicalstructure.

In embodiments, the virtual field of view can be based upon probeparameters computed in accordance with a pre-defined set of rules forthe selected view-type. The probe parameters can be computed byevaluation of cost function expressed by the pre-defined set of rulesfor the selected view-type. The predefined set of rules can vary overthe view-types in the set of view-types.

In embodiments, the virtual field of view can be determined from probeparameters, which include a probe location, a view direction and a planeorientation.

The method can further include selectively adjusting probe parameters ofthe probe, recalculating a virtual field of view of the probe based uponthe adjusted probe parameters, and rendering for display another virtualintracavity image based upon the imaging dataset of the patient and therecalculated virtual field of view of the probe.

The method can also further include selectively storing datarepresenting the virtual intracavity image and the corresponding probeparameters as part of a plan.

In embodiments, the imaging dataset of the patient can be acquired usinga volumetric imaging modality selected from the group consisting ofX-ray CT imaging, rotational angiography, Mill, SPECT, PET,three-dimensional ultrasound, and the like.

In another aspect, a method of medical intervention on patient isprovided that involves an intracavity probe, such as a TEE orintravascular probe. The method includes storing data representing atleast one virtual intracavity image and corresponding probe parametersof the probe as part of a plan for the medical intervention. With theprobe located and oriented to correspond to certain probe parametersstored as part of the plan, a live intracavity image is acquired byoperation of the probe. A display is generated that displays together avirtual intracavity image stored as part of the plan and the liveintracavity image.

In another aspect, devices, program products and methods are consideredthat store data representing at least one virtual intracavity image andcorresponding probe parameters of the probe as part of a plan for themedical intervention. With the probe located and oriented to correspondto certain probe parameters stored as part of the plan, a liveintracavity image is acquired by operation of the probe. A display isgenerated that displays together a virtual intracavity image stored aspart of the plan and the live intracavity image.

Embodiments also relate to a system for planning a medical interventionon patient that involves an intracavity probe, where the system includesmemory configured to store an imaging dataset of a patient and at leastone processor. When executing program instructions stored in the memory,the at least one processor is configured to execute one or more steps ofthe method according to embodiments herein. In a specific embodiment theat least one processor is configured to access the imaging dataset ofthe patient, select a view-type from a defined set of view-types,determine a virtual field of view of the intracavity probe correspondingto the selected view-type, and render for display a virtual intracavityimage based upon the imaging dataset of the patient and the virtualfield of view of the intracavity probe.

The system may further comprise an imaging acquisition subsystem and anintracavity probe such as a TEE probe or ICE probe. The imagingacquisition subsystem can be configured to acquire images from theintracavity probe.

The system may further comprise a volumetric imaging acquisitionsubsystem that is configured to acquire the imaging dataset of thepatient. The volumetric imaging acquisition subsystem may advantageouslyuse a volumetric imaging modality selected from the group consisting ofX-ray CT imaging, rotational angiography, SPECT, PET, three-dimensionalultrasound, and the like.

The imaging acquisition subsystems may be part or may be interfaced to amore general system for medical intervention planning. In anadvantageous configuration, it is the imaging acquisition system thatcomprises medical intervention planning capability, for exampleincluding memory and processors, either dedicated or of the generalpurpose type that are configured to perform the method steps accordingto embodiments herein. Such imaging acquisition system can equivalentlybe either the volumetric acquisition system or the intracavityacquisition system depending on the circumstances and the availabilityof processing devices. This will allow to manufacture a very compactsystem.

Other aspects and improvements are described and claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The characteristics of the invention and the advantages derivedtherefrom will be more apparent from the following description ofnon-limiting embodiments, illustrated in the annexed drawings.

FIG. 1 shows an exemplary flow chart of an embodiment of the presentapplication.

FIG. 2 shows a functional block diagram of an exemplary X-ray CT system.

FIG. 3 shows a Multi Plane Reformatting of a four-chamber view.

FIG. 4 show an example of a planned TEE within an MPR reconstruction,simulated angio view and a volume render view.

FIG. 5 shows a schematic representation used for planning for placementof a MitraClip.

FIG. 6 illustrates the possible movements to manipulate the probe toacquire a TEE image.

FIG. 7 shows an example of extracting a plane from a closed curved.

FIG. 8 shows a schematic representation of the virtual field of view.

FIG. 9 shows a screen layout of the planned TEE view.

FIG. 10 shows an exemplary flow chart for computing the TEE probeparameters and the virtual field of view.

FIG. 11 shows an example user interface with controls to adjust theprobe parameters.

FIG. 12 shows a high-level block diagram of an example of an X-ray CTsystem.

FIG. 13 shows an exemplary flow chart for acquiring a TEE image based onthe TEE plan.

FIG. 14 shows an example of a mimic TEE image based on volumetric imagedata.

FIG. 15 shows an example for manual matching the planned TEE view withan acquired TEE view using the aortic valve.

FIG. 16 shows an extended method for registering the planned TEE viewwith an acquired TEE.

FIG. 17 illustrates an example of a TEE system.

FIG. 18 illustrates possible movements to manipulate the probe toacquire an intracardiac echocardiography image.

FIG. 19 illustrates an example of an integrated system including avolumetric and an intracavity imaging subsystem according to embodimentsherein.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present application is particularly advantageous in pre-operatedplanning of intracavity imaging (such as TEE or intracardiac imaging)during transcatheter heart procedures based on patient specific CT imagedataset as acquired with a CT system and it will mainly be disclosedwith reference to this field, particularly for planning for structuralheart procedures for instance heart valve replacement, valve repair andleft atrium appendix (LAA) closures. An intracavity probe is an imagingdevice, for example of the ultrasound type, that, inserted in a cavityor orifice of the body (such as, for example, the esophagus, the rectum,the vagina, a vessel (artery or vein), heart atrium, heart ventricle,etc.) and provides images therefrom.

FIG. 1 shows a flow chart illustrating the operations according to anembodiment of the present application. The operations employ an imagingsystem capable of acquiring and processing volumetric images, forinstance computed tomography, of an organ (or portion thereof) or otherobject of interest.

FIG. 2 is a functional block diagram of an exemplary X-ray CT system,which can be used for the imaging system that is part of the operationsof FIG. 1. The exemplary X-ray CT system includes a CT imaging apparatus212 that operates under commands from user interface module 216 and willprovide data to data processing module 214.

The X-ray CT imaging apparatus 212 captures a CT scan of the organ ofinterest. The X-ray CT imaging apparatus 212 typically includes an X-raysource and detector mounted in a rotatable gantry. The gantry providesfor rotating the X-ray source and detector at a continuous speed duringthe scan around the patient who is supported on a table between theX-ray source and detector.

The data processing module 214 may be realized by a personal computer,workstation or other computer processing system. The data processingmodule 214 processes the CT scan captured by the X-ray CT imagingapparatus 212 to generate data as described herein.

The user interface module 216 interacts with the user and communicateswith the data processing module 214. The user interface module 216 caninclude different kinds of input and output devices, such as a displayscreen for visual output, a touch screen for touch input, a mousepointer or other pointing device for input, a microphone for speechinput, a speaker for audio output, a keyboard and/or keypad for input,etc. The data processing module 214 and the user interface module 216cooperate to carry out the operations of FIG. 1 as described below.

The operations of FIG. 1 can also be carried out by software code thatis embodied in a computer product (for example, an optical disc or otherform of persistent memory such as a USB drive or a network server). Thesoftware code can be directly loadable into the memory of a dataprocessing system for carrying out the operations of FIG. 1.

An embodiment is now disclosed with reference to FIG. 1. Thetherein-depicted steps can, obviously, be performed in any meaningfullogical sequence and can be omitted in parts. As it is an objective ofthe embodiments herein to provide a select (e.g. optimal) workflow thatcan be used for planning the TEE imaging during transcatheter heartprocedures based on patient specific CT image dataset, workflow examplesteps will also be referenced.

As can be seen in FIG. 1, the workflow comprises of number of steps.First patient specific image data is obtained as described in step 101of FIG. 1. The patient specific image data represents a volumetric imagedataset such as for instance obtained with a CT scanner. The patientspecific image dataset may also consist of four-dimensional (4D) data,which is a time sequence of three-dimensional (3D) that depict thecardiac motion.

In step 102 of FIG. 1, the path of the esophagus is segmented within thepatient specific image dataset. This path represents the 3D centerlineof the esophagus. For determining this centerline, similar techniquescan be applied as those used for segmenting blood vessels centerlines,being manual delineation, or automatic as for instance disclosed byGrosgeorge et al, “Esophagus Segmentation from 3D CT Data Using SkeletonPrior-Based Graph Cut”, Comput Math Methods Med. 2013; 2013: 547897.

In step 103 of FIG. 1 one or more anatomical structures (landmarks) areidentified and segmented. By using the volumetric image data, it ispossible to label and segment (or delineate) one or more anatomicalstructures using manual, semi-automatic or fully automatic methods. Anexample of manual segmentation of a mitral valve is provided byThériault-Lauzier et al, “Quantitative multi-slice computed tomographyassessment of the mitral valvular complex for transcatheter mitral valveinterventions part 1: systematic measurement methodology andinter-observer variability”, EuroIntervention. 2016 Oct. 10; 12(8):e1011-e1020. An example of (semi) automatic segmentation of the fourchambers within the heart is provided by Zhen et al, “Four-Chamber HeartModeling and Automatic Segmentation for 3D Cardiac CT Volumes UsingMarginal Space Learning and Steerable Features”, IEEE Trans Med Imaging.2008 November; 27(11):1668-81. Another example of semi-automaticsegmentation of a heart valve employs user-provided seed point(s).

Table 1 provides some examples of anatomical structures (landmarks) withtheir corresponding structure types that support in the determination ofTEE views as performed by step 105 of FIG. 1. Some of the structuretypes are simple 3D locations (Points), some are closed 3D curves(Closed curve) and some are segmented 3D structures (e.g., tube-likestructure).

TABLE 1 Examples of landmarks with their corresponding structure typeLandmark Structure Type Mitral annulus Closed curve Tricuspid annulusClosed curve Aortic annulus Close curve Pulmonary valve Tube likestructure Fossa Ovalis Closed curve Apex Point Lower Mitral CommissurePoint Upper Mitral Commissure Point Atrial appendage Tube like structureSVC (Superior Vena Cava) Tube like structure IVC (Inferior Vena Cava)Tube like structure

All of the above anatomical structures can be segmented manually and/orsemi-automatically. For the calculation of the TEE probe parameters(step 105), it does not matter if the landmarks are segmentedautomatically, semi-automatically or manually. For example, the mitralannulus landmark is often manually delineated by placing a collection ofuser defined points as for instance described by Blanke et al, “MitralAnnular Evaluation with CT in the Context of Transcatheter Mitral ValveReplacement”, JACC Cardiovasc Imaging 2015 May; 8(5):612-615. As can beseen from the Table 1, some landmarks are points, some are closedcurves, and some are tube like structures.

An example that depicts how these anatomical structures (landmarks) canbe used is shown in FIG. 3. FIG. 3 shows a Multi Plane Reformatting(MPR) of a four-chamber view based on the patient specific volumetricimage data. This four-chamber view is defined by a 2D image plane whichis based on identified landmarks. This image plane is chosen to cutthrough the center of the mitral valve 303, the tricuspid valve 304 andfossa ovalis 305, as well as the apex of the heart 306. The planned TEEprobe location 301 and virtual field of view 302 are superimposed withinthis MPR view. In this example, some of the target structures are pointsand closed curves and the image plane intersects with the centers ofthese structures. The centers of the closed curved structures can becalculated by computing the center of gravity of the 3D closed curved. Asecond example is provided by FIG. 4.

FIG. 4 shows an MPR reconstruction 401, simulated angio view 402 and avolume render view 403. Within all these three views, the TEE probelocation 404, closed curve 405 that indicates the aortic valve annulusand the TEE virtual field of view 406 are shown. In this example, thefield of view is defined by a plane calculated from the 3D closed curve405.

If the operator is preparing for an interventional procedure, he/shemight place virtual devices (replacement valves, repair devices and ordelivery devices) that will be placed or are temporarily present duringthe procedure. The locations for these devices will be estimated basedon the segmentations and the content on the images. An example of such adevice is the MitraClip and further described by Feldman et al in“Percutaneous mitral repair with the MitraClip system: safety andmidterm durability in the initial EVEREST (Endovascular ValveEdge-to-Edge Repair Study) cohort”, J Am Coll Cardiol. 2009 Aug. 18;54(8):686-94.

FIG. 5 shows a schematic representation used for planning for placementof a MitraClip. The image shows the mitral valve, with the location ofthe MitraClip device 501 the physician wants to place during theprocedure. Intersection line 502 shows where the TEE image view normallyintersects the mitral valve. During the procedure, this intersectionshould be at location 503, such that the mitral clip 501 is visible inthe TEE image. The locations of these devices can also be used tocalculate additional device specific views.

The operator (for instance, the echocardiographer) may decide that it'snot beneficial to segment heart structures. This may be necessary in thecase that the anatomy cannot be segmented automatically due to imagequality or rare anatomic variations. In this case, step 103, 104 and 105can skipped and the operator can determine the TEE probe parameters andthe virtual TEE interactively as described by step 106.

In step 104 of FIG. 1, the operator chooses a TEE view-type he/sheintends to plan. For example, a desired TEE view-type can be selectedfrom the following view-types: e.g. two-chamber view-type, four-chamberview-type, bicaval view-type, and mitral commissural view-type.

In step 105 of FIG. 1, TEE probe parameters for the desired TEEview-type can be calculated as well as the creation of the correspondingvirtual TEE view. To determine the TEE probe parameters and the virtualTEE view corresponding to the specific TEE view-type as identified instep 104, five parameters of interest can be computed. FIG. 6illustrates the five parameters of interest and are explained below:

-   -   1. Shaft insertion depth. This parameter defines the depth of        the TEE probe inside the esophagus and is archived by advancing        or withdrawal (602) of the TEE probe by the echocardiographer.        During a TEE scan, the TEE echocardiographer measures the        insertion depth from the patient's teeth. As these are rarely        included in cardiac image scans such as CT or MM, the operator        will have to calibrate against a known height. This calibration        step if further described in the workflow depicted by FIG. 13.        -   The workflow as described by FIG. 1 may allow the operator            to set the shaft insertion depth to zero at a particular TEE            view simulation (105 and/or 106). All insertion depths are            then computed relative to this depth. Such “relative depths”            can be used in the workflow as described by FIG. 13 to            simplify the depth calibration step 1302.    -   2. Shaft rotation. This parameter defines the rotation of the        shaft along its long axis (603).    -   3. Transducer rotation angle. This parameter defines the        rotation of the TEE transducer (604).    -   4. Shaft bending anterior-posterior angle. This parameter        defines the bending angle of the shaft in anterior-posterior        direction (601).    -   5. Shaft bending left-right angle. This parameter defines the        bending angle of the shaft in left-right direction (605).

The system is able to calculate, in the coordinate system of patientspecific volumetric image dataset, the virtual field of view that can beachieved by the TEE probe. The TEE probe parameters are computed basedupon the segmented esophagus path that result from step 102 of FIG. 1and the anatomical structures or landmarks (103) which describe theselected TEE view-type (104). For a specific TEE view-type, one or morerules that are associated with one or more anatomical structures aredefined for the specific TEE view-type.

Table 2 provides a few examples of TEE view-types and associated rules.

TABLE 2 Examples of view rule database (1003); overview of view rulescorresponding to specific TEE view-types. View-type Name View-type rules4 chamber Probe on Mitral Center-Apex line Mitral center in middle ofFOV Transducer rotated to show tricuspid valve center BiCaval view Probehalfway IVC and SVC Plane intersects IVC center and SVC center IVC left,SVC right on FOV ME Mitral Probe on Mitral Center-Apex line CommissuralMitral center in middle of FOV Transducer rotated that upper and lowermitral commissures are in plane Mitral Intersects Mitral valve centerShort Axis FOV approx, in plane with closed curve Center Mitral valve inField of view

Examples of common TEE view-types, as well as orientationrecommendations are described by Hahn et al, “Guidelines for performinga comprehensive transesophageal echocardiographic examination:recommendations from the American Society of Echocardiography and theSociety of Cardiovascular Anesthesiologists”, Anesth Analg. 2014January; 118(1):21-68.

Different heart structures (Table 1) and associated rules can be used indifferent ways when calculating views. In case of a point structure, therule may require that the point is positioned within the field of viewplane, and the rule can specify approximately where this point shouldappear in the field of view (for example left, middle, right). A closedcurve (like the mitral valve), can be simplified to a center point andto a plane which best fits through the closed curve by converting theclosed curve to a point cloud and then perform eigenvalue/eigenvectoranalysis on the point cloud. An example is provided by FIG. 7, whichshows a closed curve 701 that indicates the location of the mitralannulus within a volume render view. Line 702 is a line orthogonal tothe mitral annulus representing the normal vector of the plane, andlandmark 703 the center. Line 702 and landmark 703 implicitly define theannulus plane (with landmark 703 as the origin).

Rules like “Mitral Short Axis” can require the center point of themitral annulus contour to be positioned within the image plane,centered, and the field of view plane to be aligned with the planefitted (using for example least squares) through the data points. A 3Dtube-like structure can be similarly converted to a center-planecombination by calculating a center(lumen)line through the tube-likestructure and using the average position and direction of thecenterline. The direction is used as the plane normal. A center lumenline can be calculated for instance using skeletonization of thetube-like structure.

In embodiments, the transducer position and orientation of the TEE probeand the corresponding TEE Field of View of the TEE probe can becalculated by following the esophagus path for the length of theinsertion depth, and applying the rotations and bends of the TEE probe.

For example, let H be a 3×3 matrix that represents the transducerorientation system of the TEE probe, consisting of three vectors X(803), Y (802), Z (807) pointing left, forward, and up, relative to thetransducer of the TEE probe as can be seen in FIG. 8. The position L(801) represents the location of the transducer of the TEE probe. TheField of View plane (804) of the TEE probe can be identified by theposition L (801) and X (803), Y (802) vectors representing,respectively, the view direction and plane orientation.

First, the transducer orientation/location H_(d), L_(d) can bedetermined after inserting the shaft of the TEE probe down the esophagusover a distance d (602), and applying the shaft rotation ζ (603). Theesophagus path is expressed as a list of points E connected by linesegments. The initial transducer location L₀ is the start of theesophagus, E₀. L is translated along each line segment of E until atotal distance d is traveled along the path. H₀'s Z axis is aligned withthe first line segment of E.

For all the points on E traversed, coordinate system H is multiplied bya rotation matrix:

H _(k) =R(arccos(n _(k) ·n _(k-1)),n _(k) ×n _(k-1))·H _(k-1)  Eqn. (1)

Here, vector n_(k) is the normalized direction of line segment k, R(α,v)is a function that produces a rotation matrix that rotates over angle αaround vector v.

Next, the shaft rotation ζ of the TEE probe is applied to the transducerorientation H_(k) (the orientation after passing the last point of Etraversed) as follows:

H _(d) =R(ζ,Z _(k))·H _(k)  Eqn. (2)

Next, the anterior/posterior flex angle (φ, 601), the left/right flexangle (θ, 605) and the transducer angle α (604) are applied. Because ofthe catheter design, φ and θ both affect the probe location andorientation. The location L_(flex) and the orientation H_(flex) of theTEE probe can be calculated as:

L _(flex) =L _(d) +c·sin φ·Y _(d) +c·sin θ·X _(d) +c·(1−cos φ·cos θ)·Z_(d)  Eqn. (3a)

H _(flex) =R(θ+α,Y _(d))·R(φ,X _(d))H _(d)  Eqn. (3b)

Note that constant c is a probe constant representing the bendinglength.

The parameter L_(flex) represents the location position L (801) of theTEE probe as shown in FIG. 8. The parameter H_(flex) represents thetransducer orientation system H of the TEE probe, consisting of threevectors X (803), Y (802), Z (807) as shown in FIG. 8. As can be seen inFIG. 8, the Field of View plane (804) of the TEE probe can be identifiedby the solved for L (801), X (803), Y (802) vectors. Note that thetransducer orientation system is based on the transducer rotation angleα, which can be performed by physical rotation or electronically, butthis makes no difference as to the calculation of the location andorientation of the TEE probe transducer. Finally it must be noted thatthe calculations here are good approximations. However, in alternateembodiments, the calculations for location and orientation of the TEEprobe transducer and corresponding field of view can possibly accountfor tissue stiffness and/or probe physical properties and/or otherfactors.

FIG. 9 shows a possible screen layout. The images are rendered usingdate from dataset 101, in this case CT data. Viewport 901 shows a viewthat simulates the TEE field of view 905. It mimics the layout of a TEEconsole, including the indicator 906 of the transducer angle 604.Viewport 902 shows a simulated angio view with the esophagus path 904,probe 907 and field of view 905 indicated. Viewport 903 shows a volumerendering where part of the patient is made opaque, and the esophaguspath 904, and field of view 905 are indicated.

If flexing (601, 605) is employed, the probe origin can move away fromthe original esophagus path. In the patient, the probe does not actuallyleave the esophagus, instead, the esophagus is deformed. For the abovecalculations it is assumed that the esophagus deforms with noresistance.

FIG. 10 describes in detail the method to compute the TEE probeparameters and the virtual field of view of the TEE probe based upon theanatomical structures or landmarks (1001) that result from step 103 ofFIG. 1, the esophagus path segments (1002) that result from step 102 ofFIG. 1 and the TEE view rules database (1003) which describe theselected TEE view-type as defined by step 104 of FIG. 1.

In step 1004 of FIG. 10, a cost function is defined. For instance, ifthe virtual field of view of the TEE probe should intersect with aspecific landmark, the cost function contribution will be the lowest ifit is an exact intersection and the cost increases with larger distancefrom that specific landmark. For a desired TEE view-type as defined bystep 104 of FIG. 1, the applicable view rule set is chosen from the viewrules database 1003, expressed as a cost function. Using the segmentedesophagus path 1002 and the actual landmarks locations 1001, a costfunction can be defined for a specific view/patient combination C_(view)(1004) that depends only on the set of probe parameters P. For example,consider an example for a set of parameters P and a TEE Field of View(FoV) calculated from the set of parameters P. If there are twolandmarks L₁, L₂ that should be intersected by the TEE probe andcentered in the FoV, a cost function can be defined as follows:

Cost_(total) =W ₁*distance²(L ₁,plane_(FoV))+W ₂*distance²(L₂,plane_(FoV))+W ₃*distance²(C _(L1 L2) ,C _(FoV)),  Eqn. (4)

where C_(L1 L2) is the desired center of the view determined from themidpoint between L₁ and L₂),

-   -   C_(FoV) is the current center of the Field of View (FoV),    -   distance²(a, b) is the square of the distance between two        points, or between a point and a plane, and    -   W₁, W₂, W₃ are weight factors that indicate weighing for the        corresponding terms.        Note that W₁ and W₂ can be identical to one another, and W₃ can        be significantly smaller than W₁ and W₂. These constraints can        be useful to get the landmarks L₁ and L₂ into the plane than it        is to have the image centered correctly.

In step 1005 the optimal probe parameters are computed. For somecombination of probe parameters P_(optimal) it provides the best matchwith the view rules, which is when the cost function C_(view) has aminimal cost. This minimum can be searched for with a solver algorithm.Examples of such algorithms are gradient descent algorithm as describedby Wright in “Coordinate descent algorithms”, Mathematical programming,June 2015, Volume 151, or genetic algorithms, or brute force. For thegradient descent algorithm, the processing starts with an initial set ofprobe parameters P and then calculates the derivative of the costfunction with respect to all parameter values. The derivative of thecost function provides an indication of the direction ΔP that willdecrease the cost function the fastest. The set parameters P are updatedby adding ΔP and the process is repeated until the cost function isminimized such that is satisfies a stopping criterion.

The database of view rules 1003 may also contain a formula forcalculating an initial probe parameter set as input for the costfunction C_(view). It is also possible for the user to add constraints(fixed insertion distance, reduced range of flexing) as part of the viewrules. Instead of using an algorithm it is also possible to calculateprobe parameters by writing a computer program that determines probeparameters analytically or heuristically, using landmarks 1001 and viewrules 1003. In such a program the likely first step would be tocalculate the desired intersection plane for the field of view and findthe intersection with the line describing the esophagus path (1002).

Within step 1006, the virtual field of view of the TEE probe iscomputed. The field of view is defined as indicated in FIG. 8. The fieldof view imaging plane (804) is defined by the probe location 801, viewdirection 802 and the plane orientation 803, calculated from the set ofprobe parameters P_(optimal).

As depicted in FIG. 8, the virtual field of view (804) of the TEE probeis defined by the view depth (805) and view angle width (806), whichdescribes the wedge shape of a TEE scan. Note that during acquisition ofreal TEE image by operation of the TEE probe, these parameters can beelectronically controlled from the TEE console and determine the tissuearea to be imaged. Increasing this area reduces the frame rate and/orTEE image quality. During the calculation of the virtual field of viewof the TEE probe, these parameters can be predefined, controlled by theoperator, or automatically computed by using the landmarks (1001) andthe esophagus path (1002). An example of automatically computing theview depth (805) and view angle width (806) parameters is to includethese in the parameter set P. The algorithm can thereby suggest one ormore field of views that show the required anatomical landmarks usingthe smallest possible depth (805) and smallest possible view angle width(806), which improves the image quality of the acquired real TEE imageas described within the workflow with reference to FIG. 13.Alternatively, the automatic computation of the view depth (805) andview angle width (806) can take into consideration a predefined width ordepth as defined by the operator. For instance, the operator may definethat the depth or width has a minimum or maximum value.

Once the virtual field of view of the TEE probe has been computed, theoperator can be allowed to interactively view a virtual TEE imagecorresponding to the virtual field of view in step 106. The operator canpossibly adjust the probe parameters, if necessary. The workflow canthen recalculate the virtual field of view of the TEE probe based uponthe adjusted probe parameters as described above with respect to step105 and render another virtual TEE image corresponding to therecalculated virtual field of view as part of step 106. Such probeparameter adjustment, field of view re-calculation and rendering can berepeated for multiple iterations, if need be.

FIG. 11 shows an example user interface that allows a user tointeractively view a virtual TEE image corresponding to the virtualfield of view of the TEE probe as well as adjust the TEE probeparameters, if necessary. The controls shown are insertion distance1101, transducer angle 1102, shaft rotation angle 1103, ante-retroflexangle 1104 and left-right angle 1105. After changing the values, thevirtual field of view 905 of the TEE probe is updated and the renderingsin FIG. 9 are updated immediately.

The visualization of the virtual TEE image corresponding to the virtualfield of view can be accomplished by rendering the virtual field of view905 of the TEE probe based on the image data in the volumetric dataset103. The transducer rotation angle 906 is shown in graphic 901. It ispossible to render more realistic virtual TEE images using a colorlookup table. In embodiments, the operations can process the cubicvolume of voxels of the volumetric dataset to identify an arbitraryplane that cuts through this volume and corresponds to the virtual fieldof view of the TEE probe, and then interpolate the values for pixels ona virtual image on this plane using the image data for the voxels. Thistechnique is known as MPR or “multi-planar reformatting.” The resultingimage will have intensities that match image data for the voxels. Alookup table can be applied to the pixels of the resulting image totransform the intensities in order highlight soft tissue (muscle, fast)and make the remainder black. The lookup table operation can berepresented as follows:

I=T[val],  Eqn. (5)

where I is the pixel intensity varying from 0.0 to 1.0, corresponding toblack and bright white;

-   -   T is the lookup table; and    -   val is the input value of the pixel for the image derived from        the multi-planar reformatting        Note that the input value for the pixel can be negative. In one        example, the lookup table can contain a linear gradient from 1        to 0 for pixel input values in the range −255 to 195, and is 0.0        for other pixel input values.

FIG. 14 shows an example of a virtual TEE image (MPR image 1401) basedon the image data (101), which is rendered using a color lookup table.The TEE field of view (1402) is superimposed on the virtual TEE imagefor reference. Lastly, the virtual devices as indicated in step 103 canalso be rendered/simulated on the virtual TEE image.

The rendering can be static or use 4D CT data to show how the virtualTEE image at this location will vary due to the movement of the heart.For navigational purposes, the segmented heart structures from theprevious step can be superimposed on the virtual TEE images. Inaddition, the intersection generated can be superimposed on an MPR orvolume render 904. There are various techniques for this: intersectionlines, embedded planes in volume render, fusing the generated TEE imagewith the 3D volume render, partial transparency, etc. In addition, anX-Ray angio simulation view (simulated angio view) 902 is shown, thatgives a preview of the probe as it will appear on the angio view.

When the operator is satisfied, he/she can generate a planning report orplan as described by step 107 of FIG. 1. This step can involve recordingor storing a TEE view as part of the planning report. The TEE view caninclude the current virtual TEE image and possibly the probe parametersused to generate the current virtual TEE image. Furthermore, theoperator can proceed to the next desired view-type (step 108 of FIG. 1)or save the planning report for later use.

The planning report can be used pre-procedurally or during the actualprocedure (intra-procedural). Pre-procedurally the report providesvisual information whether an intra-procedural real TEE image can beobtained anatomically and can give an estimate of expected TEE imagequality obtained during the actual procedure. For instance, for sometranscatheter mitral valve procedures the septum between the left andright atrium needs to be punctured by a catheter to get access to themitral valve. This step in the transcatheter procedure is guided bymeans of TEE. A virtual TEE image that provides guidance for thispuncture is a virtual TEE image that contains both the mitral valve aswell as the point where the septum is punctured to obtain access towardsthe mitral valve. This virtual TEE image can be used to verify thedistance between the septal puncture point and the mitral valve. Inpatients with an enlarged atrium this view is often unobtainable by TEE.Having this information upfront to the actual procedure will thereforeprovide valuable input for the physician to decide for an optimaltreatment.

Another example for use of the report pre-procedurally is to assessintra-procedural TEE image quality. For good quality TEE images, the TEEprobe needs to be as close to the tissue of interest. The report willdetermine the depth of the field of view and will give visual feedbackwhether other tissues/cardiac structures are located in front of thetissue of interest. If the tissue of interest can only be obtained witha large depth or if other structure(s) is(are) located in front of thetissue of interest, the image quality will be decreased.

During the actual TEE procedure, the operator can use the virtual TEEimages and try to obtain real TEE images in the patient that replicateone or more virtual TEE images. An embodiment for this workflow is nowdisclosed with reference to FIG. 13. During the TEE procedure, theoperator opens the TEE plan (1301) as generated within the workflowdepicted by FIG. 1, either on paper, or as an electronic document. Next,the operator calibrates the calculated distance offsets (1302) betweenthe TEE plan and the actual patient. The depth/distance values 602recorded during planning are calibrated against the scale printed on theshaft by manually registering a live TEE image with an image fromplanning, thus determining the offset between the two scales.

One example of such manual matching is by using one of the planned TEEviews from the workflow as described by FIG. 1 as a reference TEE view.For all other planned TEE views within the workflow as described by FIG.1, the computed shaft insertion depth is relative to this reference TEEview. Such reference TEE view can be defined by one or more anatomicalstructures which is relatively easy to image, for instance the aorticvalve. Such manual matching using the aortic valve is illustrated inFIG. 15. The left image shows the aortic valve (1501) on the referenceTEE view and on the right the aortic valve (1502) on the live TEE image.If “relative distances” (probe location in the esophagus) is used duringthe workflow as illustrated by FIG. 1, calibration and re-calibration isdone simply reproducing the “zero distance” view on the live TEE imageacquired by the TEE probe. To simplify this manual matching process,optionally the reference TEE view can be planned with all probe angleparameters set to 0, and the live TEE image of the probe can be acquiredwithout any rotation as well.

In step 1303, the operator selects a TEE view from the TEE plan and theprobe parameters corresponding to the selected TEE view are extractedfrom the TEE plan.

In step 1304, the operator navigates the TEE probe to the location ofthe selected TEE view using the probe parameters belonging to theselected TEE view and extracted from the TEE plan in step 1303, andacquires the live TEE image at such location. This process may berepeated (1305) as, until the planned part of TEE review is completed(1306).

FIG. 16 shows an alternative and extended registration workflow that canbe used as part of workflow of FIG. 13. Such registration workflowdisplays a number of images together, including a simulated angio view(1601) (similar to 902), a virtual TEE view (1602) for one of theplanned TEE views, a live procedural X-ray image (1603), and a live TEEview (1604) acquired by the TEE probe that is positioned at the locationand orientation corresponding to the virtual TEE view (1602).Registration of the virtual TEE view (1602) and live TEE image (1604)can be performed by matching the probe position (1605) of the simulatedangio view (1601) with the probe position (1606) on the live X-ray image(1603). During the workflow as described by FIG. 1, the simulated angioview (1601) calculates the RAO/LAO, caudal/cranial angles (1607) for anoptimal or defined c-arm positioning as for instance disclosed by U.S.Pat. No. 9,008,386B2. Alternatively, the steps 105 and/or 106 from FIG.1 can be performed during the procedure. In this case the simulatedangio view (1601) can be computed by the angulation as defined by theX-ray system (RAO/LAO, caudal/cranial angles). In this case theangulation angles (1607) will match the angulation angle of the X-rayimage.

Note that if there is minimal mismatch or difference between one or moreof the virtual TEE views (1602) of the TEE plan and the correspondinglive TEE view (1604), the TEE operator can continue acquiring the liveTEE images corresponding to the TEE plan. However, if there issignificant mismatch or difference between one or more of the virtualTEE views (1602) of the TEE plan and the corresponding live TEE view(1604), the TEE operator can abandon the TEE plan and fall back onexploratory searching for the best positions for the TEE probe.

Some TEE equipment can capture biplane images, where the biplane imageFOV is orthogonal to the original FOV. Such biplane image captureplanning can be included in the workflow as described before.

Note that the shaft of the TEE probe can also be freely rotated, butdoes not require calibration. Specifically, the probe can always beinserted into the patient such that the sensor is pointing to thepatient's anterior from the esophagus. If it has been rotated this canbe assessed from the orientation of the probe handle.

The workflow described in FIG. 13 uses a plan as input as describedwithin the workflow depicted by FIG. 1. As stated before, this plan canbe an (electronic) document, but it could also be machine readable datainput for a 3D viewer similar to the software used in the workflowdescribed in FIG. 1.

In addition to the workflow described before, a different embodiment ofthis invention can exist, in which the workflow depicted in FIG. 1,whether or not combined with the workflow in FIG. 13, can be executedduring the actual TEE procedure. In this case it is not required torecord the plan as described by step 107 of FIG. 1, but the operator candirectly apply the predicted probe parameters.

TEE is a well-established imaging technique to provide exceptionallyhigh resolution images, particularly of the left atrial morphology, themitral and the aortic valve as well as other important cardiacstructures. However, TEE is a moderately invasive procedure that incursadditional risk, cost, and patient discomfort and its application isimpeded during interventional procedures because of patient's supineposition.

Intracardiac echocardiography (ICE) provides high-imaging resolution andis routinely used during atrial fibrillation (AF) ablation proceduresfor transseptal puncture and peri procedural catheter visualization.Recently, the use of ICE to guide structural heart procedures is growing(M. Alkhouli et al, “Intracardiac Echocardiography in Structural HeartDisease Interventions”, JACC: Cardiovascular Interventions, volume 11,issue 21, November 2018).

In contrast to TEE, in which the TEE probe is inserted in the patientsesophagus, during ICE the probe is inserted inside of the patients heartthrough a vein of the patients groin (e.g. femoral vein), arm (e.g.cephalic vein), or neck (axillary vein).

The presently disclosed methods also hold for ICE, with the exceptionthat for ICE the vein needs to be segmented to guide the ICE probetoward the heart instead of the esophagus. For instance, in the casethat the ICE probe is inserted through the femoral vein towards theright atrium of the heart, the ICE probe path can extend along thefemoral vein, the inferior vena cava vein and the right atrium. Thismeans that within FIG. 1 step 102 becomes “segment vein path”. This pathrepresents the 3D centerline of the vein used to guide the ICE probetoward the heart. For determining this centerline, similar techniquescan be applied as those discussed at step 102 of FIG. 1 before.

FIG. 18 shows the ICE catheter (1801), the ICE field of view (1802) andits degrees of freedom (1803-1806). Similar to the already discussed TEEprobe (FIG. 6), the ICE catheter (1801) has shaft rotation (1803),insertion depth (1804). Left-Right flex (1805) and Antero-Retro flex(1806). These are identical to the TEE probe except that the TEE canrotate its field of view (transducer angle, 604 of FIG. 6). The initialpath of the catheter, which may be automatically or manually planned,will run through a blood vessels connected to the heart, and must beextended into the heart chambers. This path may be planned manually orautomatically.

The present disclosure mainly describes the objects of interest as theheart. The skilled person would appreciate that this teaching can beequally extended to objects. Furthermore, the present disclosure refersto CT image dataset. The skilled person would appreciate that thisteaching can be equally extended to other imaging modalities, forinstance rotational angiography, MRI, SPECT, PET, 3D Ultrasound, or thelike.

The embodiment of this disclosure can be used on a standalone system orincluded directly in, for instance, a computed tomography (CT) systemand/or a TEE or ICE imaging system and/or an X-ray system. FIG. 12illustrates an example of a high-level block diagram of a computedtomography (CT) system. In this block diagram the embodiment is includedas an example how the present embodiment could integrate in such asystem.

Portions of the system (as defined by various functional blocks) may beimplemented with dedicated hardware, analog and/or digital circuitry,and/or one or more processors operating program instructions stored inmemory.

The most common form of computed tomography is X-ray CT, but many othertypes of CT exist, such as positron emission tomography (PET) andsingle-photon emission computed tomography (SPECT).

The CT system of FIG. 12 describes an X-ray CT system. In an X-ray CTsystem an X-ray system moves around a patient in a gantry and obtainsimages. Through use of digital processing a three-dimensional image isconstructed from a large series of two-dimensional angiographic imagestaken around a single axis of rotation.

For a typical X-ray CT system 120 an operator positions a patient 1200on the patient table 1201 and provides input for the scan using anoperating console 1202. The operating console 1202 typically consists ofa computer, a keyboard/foot paddle/touchscreen and one or multiplemonitors.

An operational control computer 1203 uses the operator console input toinstruct the gantry 1204 to rotate but also sends instructions to thepatient table 1201 and the X-ray system 1205 to perform a scan.

Using a selected scanning protocol selected in the operator console1202, the operational control computer 1203 sends a series of commandsto the gantry 1204, the patient table 1201 and the X-ray system 1205.The gantry 1204 then reaches and maintains a constant rotational speedduring the entire scan. The patient table 1201 reaches the desiredstarting location and maintains a constant speed during the entire scanprocess.

The X-ray system 1205 includes an X-ray tube 1206 with a high voltagegenerator 1207 that generates an X-ray beam 1208.

The high voltage generator 1207 controls and delivers power to the X-raytube 1206. The high voltage generator 1207 applies a high voltage acrossthe vacuum gap between the cathode and the rotating anode of the X-raytube 1206.

Due to the voltage applied to the X-ray tube 1206, electron transferoccurs from the cathode to the anode of the X-ray tube 1206 resulting inX-ray photon generating effect also called Bremsstrahlung. The generatedphotons form an X-ray beam 1208 directed to the image detector 1209.

An X-ray beam 1208 consists of photons with a spectrum of energies thatrange up to a maximum determined by among others the voltage and currentsubmitted to the X-ray tube 1206.

The X-ray beam 1208 then passes through the patient 1200 that lies on amoving table 1201. The X-ray photons of the X-ray beam 1208 penetratethe tissue of the patient to a varying degree. Different structures inthe patient 1200 absorb different fractions of the radiation, modulatingthe beam intensity.

The modulated X-ray beam 1208′ that exits from the patient 1200 isdetected by the image detector 1209 that is located opposite of theX-ray tube.

This image detector 1209 can either be an indirect or a direct detectionsystem.

In case of an indirect detection system, the image detector 1209consists of a vacuum tube (the X-ray image intensifier) that convertsthe X-ray exit beam 1208′ into an amplified visible light image. Thisamplified visible light image is then transmitted to a visible lightimage receptor such as a digital video camera for image display andrecording. This results in a digital image signal.

In case of a direct detection system, the image detector 1209 consistsof a flat panel detector. The flat panel detector directly converts theX-ray exit beam 1208′ into a digital image signal.

The digital image signal resulting from the image detector 1209 ispassed to the image generator 1210 for processing. Typically, the imagegeneration system contains high-speed computers and digital signalprocessing chips. The acquired data are preprocessed and enhanced beforethey are sent to the display device 1202 for operator viewing and to thedata storage device 1211 for archiving.

In the gantry the X-ray system is positioned in such a manner that thepatient 1200 and the moving table 1201 lie between the X-ray tube 1206and the image detector 1209.

In contrast enhanced CT scans, the injection of contrast agent must besynchronized with the scan. The contrast injector 1212 is controlled bythe operational control computer 1203.

An embodiment of the present application is implemented by the X-ray CTsystem 120 of FIG. 12 as follows. A clinician or other user acquires aCT scan of a patient 1200 by selecting a scanning protocol using theoperator console 1202. The patient 1200 lies on the adjustable table1201 that moves at a continuous speed during the entire scan controlledby the operational control computer 1203. The gantry 1204 maintains aconstant rotational speed during the entire scan

Multiple two-dimensional X-ray images are then generated using the highvoltage generator 1207, the X-ray tube 1206, the image detector 1209 andthe digital image generator 1210 as described above. This image is thenstored on the hard drive 1211. Using these X-ray images, athree-dimensional image is constructed by the image generator 1210.

The general processing unit 1215 uses the three-dimensional image toperform workflow as described by FIG. 1.

Another embodiment of the present application is implemented by the TEEimaging system. FIG. 17 illustrates an example of a TEE system.Referring to FIG. 18, a transesophageal (TEE) imaging system (10)includes a transesophageal probe or TEE probe (12) with a probe handle(14), connected by a cable (16), a strain relief (17), and a connector(18) to an electronics box (20). The electronics box (20) is interfacedwith a keyboard (22) and provides imaging signals to a video display(24). The electronics box (20) includes a transmit beamformer, a receivebeamformer, and an image generator. The transesophageal probe (12) has adistal part (30) connected to an elongated semi-flexible body (36). Theproximal end of elongated part (36) is connected to the distal end ofprobe handle (14). Distal part (30) of probe (12) includes a rigidregion (32) and a flexible region (34), which is connected to the distalend of elongated body (36). Probe handle (14) includes a positioningcontrol (15) for articulating flexible region (34) and thus orientingrigid region (32) relative to tissue of interest. Elongatedsemi-flexible body (36) is constructed and arranged for insertion intothe esophagus. The entire insertion tube is about 110 cm long and hasabout 30 French in diameter.

A clinician or other user acquires a TEE scan of a patient using the TEEimaging system (10) by selecting a scanning protocol using the keyboard(22) and the transesophageal probe (12). The electronics box (20) usesthe report as generated by the workflow in FIG. 1 to support theworkflow as described by FIG. 13. This described embodiment can also beimplemented by the ICE imaging system.

Another embodiment of the present application is implemented by the TEEimaging system. Within this embodiment the electronics box (20) uses theimage data from the CT system to perform the workflow as described byFIG. 1 and/or the workflow as described by FIG. 13. This describedembodiment can also be implemented by the ICE imaging system.

Another embodiment of the present application is implemented by the TEEimaging system. Within this embodiment the electronics box (20) uses theprobe parameters as computed within the workflow as described by FIG. 1and automatically sets the probe position by controlling the positioningcontrol (15). This can be for instance performed by using electronic(servo) motor, which controls the controls on the position control (15).This described embodiment can also be implemented by the ICE imagingsystem.

FIG. 19 shows an integrated system for planning a medical interventionon patient that involves an intracavity probe. The system includesmemory 1902 configured to store an imaging dataset of a patient, and atleast one processor 1901. When executing program instructions stored inthe memory, the at least one processor can be configured to execute oneor more steps of the method according to embodiments herein.

The system may advantageously comprise a volumetric imaging acquisitionsubsystem 1900 and an intracavity probe acquisition subsystem 1903. Theintracavity probe acquisition subsystem 1903, for example a TEE or anintravascular system, can be configured to acquire images from anintracavity probe 1904 interfaced thereof, while the volumetric imagingacquisition subsystem 1900 can be configured to acquire the imagingdataset of the patient using, for example, a volumetric imaging modalityselected from the group consisting of X-ray CT imaging, rotationalangiography, MRI, SPECT, PET, three-dimensional ultrasound, and the like

The at least one processor 1901 may be advantageously configured to:

-   -   store data representing at least one virtual intracavity image        and corresponding probe parameters of the intracavity probe as        part of a plan for the medical intervention;    -   with the intracavity probe located and oriented to correspond to        certain probe parameters stored as part of the plane, acquire a        live intracavity image by operation of the intracavity probe;        and    -   generate a display that displays together a virtual intracavity        image stored as part of the plan and the live intracavity image        probe acquired by operation of the intracavity probe

There have been described and illustrated herein several embodiments ofa method and system for intervention planning.

While particular embodiments of the present application have beendescribed, it is not intended that the present application be limitedthereto, as it is intended that the present application be as broad inscope as the art will allow and that the specification be read likewise.

For example, the data processing operations can be performed offline onimages stored in digital storage, such as a picture archiving andcommunication system (PACS) commonly used in the medical imaging arts.It will therefore be appreciated by those skilled in the art that yetother modifications could be made to the provided application withoutdeviating from its spirit and scope as claimed.

The embodiments described herein may include a variety of data storesand other memory and storage media as discussed above. These can residein a variety of locations, such as on a storage medium local to (and/orresident in) one or more of the computers or remote from any or all ofthe computers across the network. In a particular set of embodiments,the information may reside in a storage-area network (“SAN”) familiar tothose skilled in the art.

Similarly, any necessary files for performing the functions attributedto the computers, servers or other network devices may be stored locallyand/or remotely, as appropriate.

Where a system includes computerized devices, each such device caninclude hardware elements that may be electrically coupled via a bus,the elements including, for example, at least one central processingunit (“CPU” or “processor”), at least one input device (e.g., a mouse,keyboard, controller, touch screen or keypad) and at least one outputdevice (e.g., a display device, printer or speaker). Such a system mayalso include one or more storage devices, such as disk drives, opticalstorage devices and solid-state storage devices such as random-accessmemory (“RAM”) or read-only memory (“ROM”), as well as removable mediadevices, memory cards, flash cards, etc.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.) and working memory asdescribed above.

The computer-readable storage media reader can be connected with, orconfigured to receive, a computer-readable storage medium, representingremote, local, fixed and/or removable storage devices as well as storagemedia for temporarily and/or more permanently containing, storing,transmitting and retrieving computer-readable information. The systemand various devices also typically will include a number of softwareapplications, modules, services or other elements located within atleast one working memory device, including an operating system andapplication programs, such as a client application or web browser.

It should be appreciated that alternate embodiments may have numerousvariations from that described above. For example, customized hardwaremight also be used, and/or particular elements might be implemented inhardware, software (including portable software, such as applets) orboth.

Further, connection to other computing devices such as networkinput/output devices may be employed.

Various embodiments may further include receiving, sending, or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a computer-readable medium. Storage media and computerreadable media for containing code, or portions of code, can include anyappropriate media known or used in the art, including storage media andcommunication media, such as, but not limited to, volatile andnon-volatile, removable and non-removable media implemented in anymethod or technology for storage and/or transmission of information suchas computer readable instructions, data structures, program modules orother data, including RAM, ROM, Electrically Erasable ProgrammableRead-Only Memory (“EEPROM”), flash memory or other memory technology,Compact Disc Read-Only Memory (“CD-ROM”), digital versatile disk (DVD)or other optical storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices or any other medium whichcan be used to store the desired information and which can be accessedby the system device. Based on the disclosure and teachings providedherein, a person of ordinary skill in the art will appreciate other waysand/or methods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the presentapplication as set forth in the claims.

Other variations are within the spirit of the present disclosure. Thus,while the disclosed techniques are susceptible to various modificationsand alternative constructions, certain illustrated embodiments thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit thepresent application to the specific form or forms disclosed, but on thecontrary, the intention is to cover all modifications, alternativeconstructions and equivalents falling within the spirit and scope of thepresent application, as defined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. The terms “comprising,” “having,” “including”and “containing” are to be construed as open-ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected,” when unmodified and referring to physical connections, isto be construed as partly or wholly contained within, attached to orjoined together, even if there is something intervening.

Recitation of ranges of values herein are merely intended to serve as ashorthand method of referring individually to each separate valuefalling within the range, unless otherwise indicated herein and eachseparate value is incorporated into the specification as if it wereindividually recited herein. The use of the term “set” (e.g., “a set ofitems”) or “subset” unless otherwise noted or contradicted by context,is to be construed as a nonempty collection comprising one or moremembers.

Further, unless otherwise noted or contradicted by context, the term“subset” of a corresponding set does not necessarily denote a propersubset of the corresponding set, but the subset and the correspondingset may be equal.

Operations of processes described herein can be performed in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context. Processes described herein (or variationsand/or combinations thereof) may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs or one or more applications) executing collectively onone or more processors, by hardware or combinations thereof. The codemay be stored on a computer-readable storage medium, for example, in theform of a computer program comprising a plurality of instructionsexecutable by one or more processors. The computer-readable storagemedium may be non-transitory.

Preferred embodiments of this disclosure are described herein, includingthe best mode known to the inventors for carrying out the presentapplication. Variations of those preferred embodiments may becomeapparent to those of ordinary skill in the art upon reading theforegoing description. The inventors expect skilled artisans to employsuch variations as appropriate and the inventors intend for embodimentsof the present disclosure to be practiced otherwise than as specificallydescribed herein.

Accordingly, the scope of the present disclosure includes allmodifications and equivalents of the subject matter recited in theclaims appended hereto as permitted by applicable law. Moreover, anycombination of the above-described elements in all possible variationsthereof is encompassed by the scope of the present disclosure unlessotherwise indicated herein or otherwise clearly contradicted by context.

What is claimed is:
 1. A method of planning a medical intervention onpatient that involves intracavity probe, the method comprising:providing or accessing an imaging dataset of the patient; selecting aview-type from a defined set of view-types; determining a virtual fieldof view of the intracavity probe corresponding to the selectedview-type; and rendering for display a virtual intracavity image basedupon the imaging dataset of the patient and the virtual field of view ofthe intracavity probe.
 2. A method according to claim 1, wherein: thevirtual field of view is based upon segmentation of an intracavity pathof the probe and at least one anatomical structure.
 3. A methodaccording to claim 2, wherein: the intracavity path of the probe isselected from the group consisting of: esophagus, rectum, vagina,vessel, heart atrium, heart ventricle.
 4. A method according to claim 2,wherein: the at least one anatomical structure selected from the groupconsisting of: Mitral annulus, Tricuspid annulus, Aortic annulus,Pulmonary valve, Fossa Ovalis, Apex, Lower Mitral Commissure, UpperMitral Commissure, Atrial appendage, SVC (Superior Vena Cava), IVC(Inferior Vena Cava).
 5. A method according to claim 1, wherein: theview types are selected from the group consisting of: two chamber, fourchamber, BiCaval, Mitral Commissural, Mitral Short Axis.
 6. A methodaccording to claim 1, wherein: the virtual field of view is based uponprobe parameters computed in accordance with a pre-defined set of rulesfor the selected view-type.
 7. A method according to claim 6, wherein:the probe parameters are selected from the group consisting of: shaftinsertion depth, shaft rotation, probe rotation angle, shaft bendinganterior-posterior angle, shaft bending left-right angle.
 8. A methodaccording to claim 6, wherein: the probe parameters are computed byevaluation of cost function expressed by the pre-defined set of rulesfor the selected view-type.
 9. A method according to claim 8, wherein:the cost function depends on weighted distance parameters between atleast one anatomical structure and an optimal field of view plane orpoints thereof.
 10. A method according to claim 6, wherein: thepredefined set of rules vary over the view-types in the set ofview-types; and the probe parameters depend on the set of rules of theselected view type, the anatomical structure and the intracavity path ofthe probe.
 11. A method according to claim 6, wherein: the virtual fieldof view is determined from the probe parameters, which include a probelocation and a probe orientation, wherein the probe orientation isdefined by a view direction and a plane orientation.
 12. A methodaccording to claim 1, wherein: the virtual field of view is based uponuser input preferably without segmentation of an intracavity path of theprobe.
 13. A method according to claim 1, wherein: the intracavity probecomprises a TEE probe or ICE probe.
 14. A method according to claim 1,further comprising: selectively adjusting probe parameters of theintracavity probe; recalculating a virtual field of view of the probebased upon the adjusted probe parameters; and rendering for displayanother virtual intracavity image based upon the imaging dataset of thepatient and the recalculated virtual field of view of the probe.
 15. Amethod according to claim 1, further comprising: selectively storingdata representing the virtual intracavity image and the correspondingprobe parameters as part of a plan.
 16. A method according to claim 1,wherein: the imaging dataset of the patient is acquired using avolumetric imaging modality selected from the group consisting of X-rayCT imaging, rotational angiography, MRI, SPECT, PET, three-dimensionalultrasound, and the like.
 17. A method of medical intervention onpatient that involves an intracavity probe, the method comprising:storing data representing at least one virtual intracavity image andcorresponding probe parameters of the intracavity probe as part of aplan for the medical intervention; with the intracavity probe locatedand oriented to correspond to certain probe parameters stored as part ofthe plan, acquiring a live intracavity image by operation of theintracavity probe; and generating a display that displays together avirtual intracavity image stored as part of the plan and the liveintracavity image.
 18. A method according to claim 17, wherein: theintracavity probe comprises a TEE probe or ICE probe.
 19. A system forplanning a medical intervention on patient that involves an intracavityprobe, the system comprising: memory configured to store an imagingdataset of a patient; and at least one processor that, when executingprogram instructions stored in the memory, is configured to: access theimaging dataset of the patient, select a view-type from a defined set ofview-types, determine a virtual field of view of the intracavity probecorresponding to the selected view-type, and render for display avirtual intracavity image based upon the imaging dataset of the patientand the virtual field of view of the intracavity probe.
 20. A systemaccording to claim 19, further comprising: an imaging acquisitionsubsystem and an intracavity probe such as a TEE probe or ICE probe, theimaging acquisition subsystem being configured to acquire images fromthe intracavity probe.
 21. A system according to claim 20, wherein theat least one processor is configured to: store data representing atleast one virtual intracavity image and corresponding probe parametersof the intracavity probe as part of a plan for the medical intervention;with the intracavity probe located and oriented to correspond to certainprobe parameters stored as part of the plan, acquire a live intracavityimage by operation of the intracavity probe; and generate a display thatdisplays together a virtual intracavity image stored as part of the planand the live intracavity image.
 22. A system according to claim 19,further comprising: an imaging acquisition subsystem that is configuredto acquire the imaging dataset of the patient.
 23. A system according toclaim 22, wherein: the imaging acquisition subsystem uses a volumetricimaging modality selected from the group consisting of X-ray CT imaging,rotational angiography, MRI, SPECT, PET, three-dimensional ultrasound,and the like.
 24. A method of planning a medical intervention on patientthat involves intracavity probe, the method comprising: providing oraccessing an imaging dataset of the patient; determining an intracavitypath of the intracavity probe; determining probe parameters; determininga virtual field of view of the intracavity probe based on theintracavity path and the probe parameters; and rendering for display avirtual intracavity image based upon the imaging dataset of the patientand the virtual field of view of the intracavity probe.
 25. A methodaccording to claim 24, wherein: the intracavity path is determined basedon segmentation of at least one cavity structure selected from the groupconsisting of: esophagus, rectum, vagina, vessel, heart atrium, or heartventricle.
 26. A method according to claim 24, wherein: the probeparameters are computed in accordance with a pre-defined set of rulesfor a view-type selected from a defined set of view-types.
 27. A methodaccording to claim 24, wherein: the probe parameters are determinedinteractively through user input.
 28. A method according to claim 24,wherein: the probe parameters are selected from a group consisting of:shaft insertion depth, shaft rotation, probe rotation angle, shaftbending anterior-posterior angle, shaft bending left-right angle.
 29. Amethod according to claim 24, wherein: the virtual field of view isdetermined from the probe parameters, which include a probe location anda probe orientation, wherein the probe orientation is defined by a viewdirection and a plane orientation.
 30. A method according to claim 24,wherein: the intracavity probe comprises a TEE probe or ICE probe.
 31. Amethod according to claim 24, further comprising: selectively adjustingprobe parameters of the intracavity probe; recalculating a virtual fieldof view of the probe based upon on the intracavity path and the adjustedprobe parameters; and rendering for display another virtual intracavityimage based upon the imaging dataset of the patient and the recalculatedvirtual field of view of the probe.
 32. A method according to claim 24,further comprising: selectively storing data representing the virtualintracavity image and the corresponding probe parameters as part of aplan.
 33. A method according to claim 24, wherein: the imaging datasetof the patient is acquired using a volumetric imaging modality selectedfrom the group consisting of X-ray CT imaging, rotational angiography,MRI, SPECT, PET, three-dimensional ultrasound, and the like.